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Abstract 

A trellis is a graph associated with a linear code that 
is used for maximum-likelihood decoding. The decoding 
complexity of a linear code is strongly influenced by the 
state complexity of the trellis, which highly depends on the 
coordinate permutation of the linear code. The problem of 
finding the coordinate permutation of a linear code such 
that the state-complexity of the associated trellis is at 
most k has been referred to as the Art of Trellis Decoding 
and is NP-hard. We show that it is linear-time solvable 
if k is constant. To this end, we reduce the problem to 
Hypergraph Cutwidth, which we show to be fixed- 
parameter linear by, for the first time, applying an analog 
of the Myhill-Nerode theorem from formal language theory 
to a hypergraph problem. In contrast, we also exploit 
Myhill-Nerode methods to show that for instances whose 
incidence graphs have bounded treewidth, Hypertree 
Width and variants cannot be solved by simple finite tree 
automata. 

1 Introduction 

In coding theory, a linear block code is an error-correcting 
code where each codeword is a vector in a fixed vector space 
over a fixed finite field such that every linear combination 
of codewords is also a codeword. After a transmission of 
data over a noisy or unreliable communication channel, 
decoding algorithms — such as the Viterbi algorithm [33]— 
are used to restore the original message. The complexity 
of many such decoding algorithms heavily depends on the 
size and structure of a directed graph associated to the 
code, its minimal trellis. 

A trellis of a linear block code is a directed graph with 
labeled arcs, one source vertex and one sink vertex. Every 
directed path from the source vertex to the sink vertex 
corresponds to the codeword consisting of the sequence of 
arc-labels encountered on this path, and each codeword of 
the code corresponds to at least one such path. Moreover, 
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every arc and every vertex of the trellis occur on at least one 
path from the source to the sink. Since a trellis represents a 
linear block code, all paths from the source to the sink have 
the same length. A vertex at distance i from the source is 
said to have time index i. The state-complexity of a trellis 
is the maximum number of vertices at any time index. The 
minimal trellis of a linear block code is the trellis such 
that any other trellis has at least as many vertices at time 
index i for every i, < i < n. Trellises were introduced 
by Forney [12] in 1967 in studying the Viterbi algorithm. 
For an overview of trellis-related decoding algorithms and 
further applications of trellises, sec the survey of Vardy [32]. 

A generator matrix for a linear block code is a matrix 
whose rows form a basis of the codeword vector space. 
If wc permute the columns of the generator matrix, we 
obtain another code, which is equivalent to the original 
code from an error-correction perspective. However, the 
permutation may greatly influence the state-complexity of 
the minimal trellis. The problem of finding a permutation 
that minimizes the state-complexity of the minimal trellis 
was termed the "Art of Trellis Decoding" by Massey [27] 
in 1978. It has attracted a lot of attention in the coding 
theory literature [32] . The corresponding decision problem, 
Trellis Width, is to decide whether there is a permu- 
tation yielding a minimal trellis with state-complexity at 
most k. It was a long-standing open question whether 
Trellis Width is NP-complete. Forney [15] referred to 
it as the only significant open problem in the context of 
trellis decoding of linear block codes. In 2007, Trellis 
Width was finally shown NP-complete by Kashyap [22]. 

Kashyap [23] also observed that Trellis Width is 
equivalent to Matroid Pathwidth. Based on results 
and conjectures in Matroid Minor theory [18], Kashyap ex- 
pressed a strong belief that Trellis Width is polynomial- 
time solvable if k is a constant. Indeed, the class of matroids 
with pathwidth at most k is closed under taking matroid 
minors [23], has bounded branchwidth, and membership 
in a minor-closed matroid family can be tested in cubic 
time for matroids with bounded branchwidth [21]. This 



asserts that, for every constant k, there exists a cubic- 
time algorithm for Trellis Width. 1 However, the list 
of minimal forbidden minors and, hence, the algorithm, 
is only known explicitly for k — 1 [23]. For k > 1, the 
algorithm, based on well-quasiordering, is nonuniformly 
fixed-parameter tractable (there is a different algorithm 
for each k) [7], and nonconstructive: for each constant k, 
we only know that an 0(n 3 )-time algorithm exists, but it 
is unknown what it is, and what kind of dependence on k 
is hidden in the big-O. 

Our Contributions. We give an explicit algorithm for 
Trellis Width that, for every fixed constant k, runs in 
linear time. Stated in terms of Parameterized Complex- 
ity [7] , our algorithm is fixed-parameter linear. 

As a first step, we first use structural properties of mini- 
mal trellises to reduce Trellis Width to Hypergraph 
Cutwidth. To show that Hypergraph Cutwidth is 
fixed-parameter linear, we extend the graph-theoretic ana- 
log [8] of the Myhill-Nerode characterization of regular 
languages first to colored graphs, and then to hypergraphs. 
Our hypergraph analog of the Myhill-Nerode theorem can 
then be used to derive fixed-parameter linear algorithms 
for hypergraph problems on instances with bounded inci- 
dence treewidth, i.e., instances whose incidence graphs have 
bounded treewidth. Incidence treewidth is a particularly 
nice parameter since it is bounded from above by other 
well-known width measures of hypergraphs: for example, 
by the treewidth of the primal graph plus one [24] , and by 
the treewidth of the dual graph plus one [30] . 

Our fixed-parameter linear algorithm for Hypergraph 
Cutwidth improves the 0(n k + 3fc + 3 ) time algorithm of 
Miller and Sudborough [28]. Besides Hypergraph Cut- 
width, Trellis Width, and Matroid Pathwidth, our 
extensions of the Myhill-Nerode theorem to colored graphs 
and hypergraphs are likely to apply to other problems. 
Colored or annotated graphs allow for more realism in 
problem modeling and often arise as subproblems when 
solving pure graph problems. It is also straightforward to 
use our methods for annotated hypergraphs. 

The original Myhill-Nerode theorem can be used both 
positively and negatively: to show that a language is reg- 
ular, and to show that a language is not regular. As an 
example of a negative use of our hypergraph Myhill-Nerode 
analog, we prove that for instances with bounded incidence 
treewidth, the Hypertree Width, Generalized Hy- 
pertree Width, and Fractional Hypertree Width 
problems cannot be solved by bottom-up finite-state tree 
automata (shortly, finite tree automata) whose inputs are 
parse trees corresponding to bounded-width tree decompo- 
sitions of the incidence graph. Moreover, the proof leads 
us to the conjecture that these problems are not fixed- 
parameter tractable with respect to the parameter incidence 
treewidth t, that is, we conjecture that there are no algo- 
rithms for these problems running in time f{t) ■ n c , where 
n is the input size, c is a constant and / only depends on t. 



1 This has also been observed by Navin Kashyap [private commu- 
nication] . 



2 Preliminaries 

For standard graph-theoretic notions, we follow the defini- 
tions and notations of Diestel [6] . 

Graph Decompositions. A tree decomposition of a 
graph G = {V, E) is a pair ({JQ : i G /}, T) where X, C V, 
i (z I, are called bags and T is a tree with elements of / as 
nodes such that: 

1. for each edge {u, v} g E, there is an i £ I such that 
{u, v} C Xi, and 

2. for each vertex v € V, T[{i e / : v e Xi}] is a 
(connected) tree with at least one node. 

The width of a tree decomposition is maxjg/ — 1. The 
treewidth of G is the minimum width taken over all tree 
decompositions of G. The notions of path decomposition 
and pathwidth of G are defined the same way, except that 
T is restricted to be a path. 

Hypergraphs. A hypergraph H is a couple (V, E), where 
V is a set of vertices and E a multiset of hyperedges such 
that e C V for each e e E. Let H = (V, E) be a hyper- 
graph. The primal graph of H , denoted G(H), is the graph 
with vertex set V that has an edge {u, v} if there exists a 
hyperedge in H incident to both u and v. It is sometimes 
called the Gaifman graph of H. The incidence graph of H, 
denoted 1(H), is the bipartite graph (V' , E') with vertex 
set V' = V U E and for v € V and e £ E, there is an 
edge {v, e} € E' if v e e. 

Hypergraph decompositions. Let H be a hypergraph. 
The treewidth of H is the treewidth of Q (H). The incidence 
treewidth of H is the treewidth of 1(H). 

Generalized hypertree width is defined with respect to 
tree decompositions oiQ(H), however, the width of the tree 
decompositions is measured differently. Suppose H has 
no isolated vertices (otherwise, remove them). A cover of 
a bag is a set of hyperedges such that each vertex in the 
bag is contained in at least one of these hyperedges. The 
cover width of a bag is the minimum number of hyperedges 
covering it. The cover width of a tree decomposition is the 
maximum cover width of any bag in the decomposition. 
The generalized hypertree width of H is the minimum cover 
width over all tree decompositions of G(H). 

The hypertree width of H is defined in a similar way, 
except that, additionally, the tree of the decomposition is 
rooted and a hyperedge e can only be used in the cover of 
a bag Xi if Xi contains all vertices of e that occur in bags 
of the subtree rooted at the node i. 

The fractional hypertree width of H is defined in a similar 
way as the generalized hypertree width, except that it uses 
fractional covers: in a fractional cover of a bag, each 
hyperedge is assigned a non-negative weight, and for each 
vertex in the bag, the sum of the weights of the hyperedges 
incident to it is at least 1. The fractional cover width of 
the bag is the minimum total sum of all hyperedges of a 
fractional cover. 

3 Trellis Width as a Hypergraph Problem 

In this section we formally define the Trellis Width and 
the Hypergraph Cutwidth problems, and we give a pa- 
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rameterized reduction from the former to the latter. Then 
we briefly consider the related Trellis Branch Complex- 
ity problem and observe that it is fixed-parameter linear. 2 

Trellis Width. Let q be a prime power. We consider 
linear (block) codes over the finite field ¥ q . A linear code 
of length n and rank d is a linear subspace C with di- 
mension d of the vector space F™. The vectors in C are 
called codewords. Linear codes are usually represented by 
a generator matrix whose d rows are a basis of C. 

A trellis D = (V, A) for a linear code C of length n is an 
arc-labeled directed acyclic graph with one source node s 
and one sink node t with arc labels from ¥ q such that 

1. every vertex and every arc of the trellis is on at least 
one s-i-path, and 

2. the set of n-tuples obtained by reading off the arc 
labels obtained in traversing all s-i-paths is C. 

The number of arcs that need to be traversed to reach a 
vertex v from s is the time index associated with v. We 
denote by Vd{i) the set of vertices at time index i in D. 
Thus, Vb(0) = {s} and V D (n) = {t}. 

It has been shown [13, 16, 25, 29] that every linear 
code C has a unique minimal trellis, characterized by 
the fact that any other trellis for C has at least as many 
vertices at time index i, for every i, < i < n, where 
n is the length of the code. In his survey, Vardy [32] 
presents four different constructions of the minimal trellis 
that appeared in the literature [2, 13, 25, 27]. From these 
constructions, it follows that the number of vertices at each 
time index is a power of q. The state- complexity or trellis 
width of a code is maxo<i< n {si}, where q Si = |Vo(i)| and 
D is its minimal trellis. 

The decoding complexity — for example, by the Viterbi 
algorithm [33] — is closely tied to the width of the associated 
trellis. However, a permutation of the columns of a code 
may greatly influence the width of the resulting minimal 
trellis, whereas from an error-correction viewpoint the 
performance of the code remains the same. This leads us 
to the following computational problem. 

Trellis Width 

Input: A prime power q, a generator matrix A of a linear 
block code over ¥ qi and a natural number k. 

Question: Is there a column permutation of A that yields 
a code with state-complexity at most k? 

The problem was only recently shown to be NP-hard 
by Kashyap [22, 23], settling a long-standing conjecture. 
It is also equivalent to the Matroid Pathwidth 
problem [23, 32]. 

Hypergraph Cutwidth. Let H = (V, E) be a hyper- 
graph. A linear layout of H is an injective map I : V — > K 
of vertices onto the real line. The cut at position i € 1 with 
respect to I, denoted 0; is the set of hyperedges that 
contain at least two vertices v, w such that l(v) < i < l(w). 
The cutwidth of the layout I is max; e R \9i t H The 
cutwidth of the hypergraph H is the minimum cutwidth 
over all the linear layouts of H. The Hypergraph 
Cutwidth problem is then defined as follows. 

2 When we do not specify a parameter, we implicitly assume the 
standard parameterization, i.e., the parameter is what one would 
optimize in the optimization version of the problem. 



Hypergraph Cutwidth 

Input: A hypergraph H = (V, E) and a natural num- 
ber k. 

Question: Does H have cutwidth at most k? 

The problem naturally generalizes the NP-complete [17] 
Graph Cutwidth problem for which several fixed- 
parameter algorithms are known [1, 4, 9, 10, 31]. Ca- 
hoon and Sahni [5] designed algorithms for Hypergraph 
Cutwidth with k < 2, with running time 0(n) for k = 1 
and running time 0(n 3 ) for k = 2, where n is the number 
of vertices. For arbitrary k, Miller and Sudborough [28] 
designed an algorithm with running time 0(n k + 3fc + 3 ). 

Trellis Width as Hypergraph Cutwidth. Exploiting 
the structure of minimal trellises, one can reduce Trel- 
lis Width to Hypergraph Cutwidth. Let D be the 
minimal trellis for a linear code C over F g with gen- 
erator matrix A. Let n and d denote the number of 
columns and rows of A, respectively. Since the rows of A 
are a basis of C, no row consists of zeros only. For a 
row c = (ci, . . . ,c„) of A, the span of c, denoted s(c), is 
the interval [a, b] where a is the smallest integer i for which 
Ci is nonzero and b is the largest integer i for which Cj is 
nonzero. For any i, 1 < i < n, denote by Pi the number 
of rows c = (ci, . . . ,c„) of A with s(c) C [l,i], and de- 
note by fi the number of rows c — (ci, . . . , c„) of A with 
s(c) C [i + l,n]. By the State Space Theorem of Forney 
and Trott [16], we have 

State Space Theorem ([16]). 

|VbC*)l =q d - pi ~ h for i = 0,1,..., n. 

We are now ready to present a parameterized reduction [7] . 

Lemma 1. There is a linear-time parameterized reduction 
from Trellis Width to Hypergraph Cutwidth. 

Proof. Let (q, A, k) be an instance for Trellis Width. 
We construct an equivalent instance (H = (V,E),k) 
for Hypergraph Cutwidth as follows: We set V := 
. . . , v n }, where n is the number of columns of A. For 
each row (ci, . . . , c n ) of A, let there be a hyperedge e € E 
such that Vi G e <^=> cj 7^ 0. This describes all the 
hyperedges of H. Note that the parameter "trellis width" 
of the input instance equals the parameter "cutwidth" of 
the output instance and that this construction works in 
linear time. 

Suppose there is a column permutation tt of A that yields 
a code C with state-complexity at most k. Consider the lin- 
ear layout I that maps Vi to 7r(i), 1 < i < n. The cutwidth 
of / is max ia \6^ H {i)\ = max ie{li ... : „_ 1} \0^ H (i + 0.5) | . 
Let Si i7r denote the number of codewords that have a non- 
zero entry in the first i columns and a non-zero entry in 
the last n — i columns of the matrix obtained from A by 
the permutation tt of the columns. Consider an arbitrary 
i € {1, . . .,ra-l}. Then, \di, H {i+0.5)\ = s 4)ir . By the State 
Space Theorem, the minimal trellis of C has g Si 7r vertices at 
time index i. Therefore, \0i t jj(i + 0.5) | = s i 7r < k. Since i 
was arbitrary, it follows that the cutwidth of H is at most k. 

Suppose there is a linear layout I of H with cutwidth at 
most k. Without loss of generality, assume I is a bijection 
from V to {1, . . . , 71}. Consider the column permutation it 
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such that l(vi) = Let G denote the linear code 

whose generator matrix is the column permutation 7r of A. 
Again, it follows from the State Space Theorem, that the 
minimal trellis for C has state-complexity at most k. □ 

By this reduction, any fixed-parameter linear algorithm for 
Hypergraph Cutwidth translates into a fixed-parameter 
linear algorithm for Trellis Width and, therefore, also 
for Matroid Pathwidth. 

Trellis Branch Complexity. Besides minimizing the 
maximum vertex-count in the minimal trellis at any time 
index, the problem to minimize the arc-count between any 
two consecutive time indices has also been proposed in the 
literature [14]. Denote by Eu{i) the set of arcs that end 
at a vertex from Vo(i). The branch- complexity of a code 
is maxi<j<„{&j} where q bi = \Er>(i)\ and D is its minimal 
trellis. 

Trellis Branch Complexity 

Input: A prime power q, a generator matrix A of a linear 
block code over ¥ q , and a natural number k. 

Question: Is there a column permutation of A that yields 
a code with branch-complexity at most fc? 

Similarly to the State Space Theorem, the number of arcs 
between two consecutive time indices is easily read off the 
generator matrix. 

Branch Space Theorem ([14]). 

\E D (i)\ = (? «'-»-i-/i for i = 1,2,..., n. 

Replacing each non-zero entry in the generator matrix by 
a T' easily reduces Trellis Branch Complexity to 
Gate Matrix Layout. 

Gate Matrix Layout 

Input: A Boolean matrix M and an integer k. 

Question: Is there a permutation of the columns of M 
so that, if in each row we enforce the consecutive-ones 
property, then no column contains more than k ones? 

Here, enforcing the consecutive-ones property in a row 
means replacing by a T' each '0' that occurs between 
two T's. The Gate Matrix Layout problem with param- 
eter k has a parameterized reduction to Pathwidth [10]. 
Since Pathwidth is fixed-parameter linear [3] and all 
reductions are linear-time, we have the following theorem. 

Theorem 1. Trellis Branch Complexity is fixed- 
parameter linear. 

4 Myhill-Nerode Analog for Colored 
Graphs and Hypergraphs 

The aim of this section is to generalize the Myhill-Nerode 
analog for graphs [8] to colored graphs and hypergraphs. 
This will give us the tools to show that Hypergraph 
Cutwidth is fixed-parameter linear. 

The methodology is first extended to colored graphs. 
From this, we obtain a Myhill-Nerode analog for hyper- 
graphs, since every hypergraph can be represented as its in- 
cidence graph with two vertex types: vertices that represent 
hyperedges and vertices that represent hypergraph- vertices. 



In the last part of the section, we finally describe how 
our Myhill-Nerode analog yields linear-time algorithms for 
hypergraph problems. 

We follow and adapt the notation used by Downey and 
Fellows [7, Section 6.4]. 

4.1 Colored Graphs 

We now develop an analog of the Myhill-Nerode theo- 
rem for colored graphs. The original Myhill-Nerode the- 
orem is stated for languages in terms of concatenations 
of words. Hence, we clarify what concatenating colored 
graphs means. 

Definition 1. A t-boundaried graph G is a graph with 
t distinguished vertices that are labeled from 1 to t. These 
labeled vertices are called boundary vertices and the bound- 
ary, d(G), denotes the set of boundary vertices of G. 

Let Gi and G2 be t-boundaried graphs whose vertices 
are colored with colors from {1, . . . ,c max }- We say that 
G\ and G2 are color- compatible if the vertices with the 
same labels in d(Gi) and d{G2) have the same color. 

For two color-compatible i-boundaried graphs, we denote 
by G-y © c G2 the colored graph obtained by taking the 
disjoint union of G\ and G2 and identifying each vertex 
of d(G\) with the vertex of d[G%) with the same label, 
wherein vertex colors are inherited from G\ and G2. 

Let Ufl^ be the universe of {1, . . . , c max }-colored t- 

boundaried graphs and F C W t a c rge . We define the canon- 
ical right congruence ~j? for F as follows: for G\,Gi € 
Wj" ge , Gi ~ F G2 if and only if G\ and G2 are color- 
compatible and for all color-compatible H € U\ a ^ c , it 
holds that G x ® c H e F G 2 ® C H e F. 

The index of ~f is its number of equivalence classes. 

Definition 1 is illustrated in Figure 1. Two colored t- 
boundaried graphs G\ and G2 are isomorphic and we 
write Gi = G2 if there is a graph isomorphism for the 
underlying (ordinary) graphs mapping each vertex to a 
vertex with the same color. 

Before we can state our analog of the Myhill-Nerode 
theorem for colored graphs, we show that every colored 
graph of treewidth at most t can be generated using a 
constant number of graph operations. To this end, we use 
the following set of operators. For generating graphs of 
only one color, the given operators coincide with those 
given by Downey and Fellows [7, Section 6.4]. 

Definition 2. The size-(t+l) parsing operators for {1, . . . , 
Cmax}-colored graphs are: 

i) {0m,...,n CmaK : J2i=T n>i=t + 1} is a family of miliary 
operators that creates boundary vertices 1, ...,£ + 1, 
of which the first n\ vertices get color 1, the next ri2 
vertices get color 2, and so on. 

ii) 7 is a unary operator that cyclically shifts the bound- 
ary. That is, 7 moves label j to the vertex with 
label j + 1 (mod t + 1). 

iii) i is a unary operator that assigns the label 1 to the 
vertex currently labeled 2 and label 2 to the vertex 
with label 1. 

iv) e is a unary operator that adds an edge between the 
vertices labeled 1 and 2. 
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(a) A 2-colorcd 3-boundaried graph G. 
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(b) A 2-colored 3-boundaried graph H. 
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(c) The glued graph G C ff. 

Figure 1: Two color-compatible 3-boundaried graphs G 
and H and their glued graph, where the boundary vertices 
are marked by their label. 



v) {ug : 1 < £ < c max } is a family of unary operators that 
add a new vertex of color £ and label it 1, unlabeling 
the vertex previously labeled 1. 

vi) €B C is our gluing operator from Definition 1. 

Obviously, for a constant number of colors c max , the set of 
size-(i + 1) parsing operators is finite. Following the proof 
of Downey and Fellows [7, Theorem 6.72], it is easy to verify 
that the graphs generated by the operators in Definition 2 
are isomorphic precisely to the graphs of treewidth at 
most t and at least t + 1 vertices. The same proof shows 
how, from a width-t tree decomposition of a colored graph, 
a linear-size parse tree for an isomorphic graph can be 
obtained in linear time. 

Definition 3. The set denotes the set of {1, ... , 

Cmax}-colored i-boundaried graphs that can be generated 
by the operators in Definition 2. 

Theorem 2. Let F C U^™^ be a family of graphs. The 
following statements are equivalent: 

i) The parse trees corresponding to graphs in F are rec- 
ognizable by a finite tree automaton. 

ii) The canonical right congruence has finite index 
over U t s ™H . 

Our proof of Theorem 2 adapts the proof of Downey and 
Fellows [7, Theorem 6.77]. To this end, we only need to 
show that our operators satisfy the Parsing Replacement 
Property. 

Definition 4. (Parsing Replacement Property). An n-ary 
operator ® has the Parsing Replacement Property if, for 
the arguments Hi, ... , H n g W|™ a11 of ® and each i, there 



Lemma 2. The size-(t + 1) parsing operators in Defini- 
tion 2 have the Parsing Replacement Property. 

Proof. Let H £ and let %* H denote the graph that 

contains only t + 1 boundary vertices, no edges, and that is 
color-compatible with H. Finally, let % l H denote the graph 
that contains t+1 boundary vertices and one additional ver- 
tex with color £ and that is color-compatible with H. Note 
that $h,$*h € ^t.ci^IL 1 we can generate them using the op- 



erators in Definition 2 because %* H and w H have at least t+1 
vertices and treewidth at most t. Now, the lemma imme- 
diately follows, noting that for each graph H € Wf™ , 



Hi 



J H > 



* iH 

eH = H(B c e(r H ), 
u e H = H 



since = ignores boundary labels, 



and that 



and 



Replacement Property. 



trivially have the Parsing 

□ 



Having proven Lemma 2, the proof of Theorem 6.77 in the 
book of Downey and Fellows [7] also proves Theorem 2 
and we have a handy tool to generalize the Myhill-Nerode 
theorem to hypergraphs. 

4.2 Lifting to Hypergraphs 

To make the method accessible to hypergraph problems 
we lift the Myhill-Nerode theorem for colored graphs of 
the previous subsection to hypergraphs. 

Definition 5. A t-boundaried hypergraph G is a hyper- 
graph with t distinguished vertices and hyperedges labeled 
from 1 to t. Two t-boundaried hypergraphs are gluable if 
no vertex of one hypergraph has the label of a hyperedge 
of the other hypergraph. 

Let Gi and G2 be gluable t-boundaried hypergraphs. We 
denote by G\ (Bh G2 the t-boundaried hypergraph obtained 
by taking the disjoint union of G\ and G 2 , identifying each 
labeled vertex of G\ with the vertex of G2 with the same 
label, and replacing the hyperedges with label £ by the 
union of these hyperedges. 

Let -Ht arge be the universe of t-boundaried hypergraphs 
and F C "H 4 algc . We define the canonical right congru- 
ence ~p for F as follows: for G© G 2 G rif ge , Gi ~ F G 2 
if and only if for all H e n\ ais ° 7 Gi h H e F ^> 
G 2 © h H e F. 



is a graph G € such that Hi 



' H„, = Hi 



G. 



To prove a Myhill-Nerode theorem for hypergraphs, we 
still need a way to create hypergraphs from parsing 
operators, as we did using the operators from Definition 2 
for colored graphs. To this end, we indeed simply use 
the operators from Definition 2, observing that every 
bipartite graph can be interpreted as the incidence graph 
of a hypergraph. Moreover, if the two disjoint independent 
sets of a two-colored bipartite graph have distinct colors, 
then we can interpret this bipartite graph as a hypergraph 
in a unique way. 

Definition 6. A well-colored t-boundaried graph is a {1, 2}- 
colored t-boundaried graph G = (U \±iW,E), where the 
vertices in U have color f , the vertices in W have color 2, 
and where U and W are independent sets. 
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For a well-colored i-boundaried graph G — (U W W, E), 
we denote by W{G) the i-boundaried hypergraph with the 
vertex set U and the edge set {N{w) : w G W}. Moreover, 
vertices of rl(G) inherit their label from G and edges e = 
N(w),w € W of "H(G) inherit the label of w. For a set U C 
W t largc , we denote W(f7) := {if G Hj argc : H = H(G), 
G G C/}. 

The incidence graph of "H(G) is G and the incidence 
treewidth of T-L{G) is the treewidth of G. 

Obviously, every # G H\ argc is %(G) for some G G ^ la 2 rgc . 
Also, every set T C Hj argc is J 7 = "H(F) for some F C 
£/ t la 2 gc . Definitions 5 and 6 are illustrated in Figure 2. 

Theorem 3. Let T C Ul™ al1 be a set of well-colored t- 
boundaried graphs. The following statements are equiva- 
lent: 

i) The parse trees corresponding to graphs in F are rec- 
ognizable by a finite tree automaton. 

ii) The canonical right congruence ^-u(F) has finite index 
overn(U t y alt ). 

Proof. Let F := {G G U^ n : H{G) G W{F)}, that is, 
for each G G F, the set F contains additionally all t- 
boundaried graphs G' G U^ an for which H(G) = H{G'). 
Clearly, every such G' is isomorphic to G, that is, F con- 
tains F and i-boundaried graphs isomorphic to graphs 
in F. Hence, the parse trees corresponding to i-boundaried 
graphs in F are the same as those corresponding to F. 
Moreover, T-l(F) = T-l(F). Hence, it remains to show that 
~H(F) nas nn ite index over H^f™ 3,11 ) if and only if ^p 
has finite index over Ul™ , 

Assume that ~p has infinite index over W t SI 2 a11 . Then 
there is a family G 1} G 2 , G 3 , . . . of graphs in W t s m a U that 
are pairwise inequivalent under ^p. Since there are only 
a finite number of possibilities to assign two colors to 
t boundary vertices, there are an infinite number of color- 
compatible graphs among Gi, G%, . . . . Moreover, notice 
that all Gi that are not well-colored i-boundaried graphs 
are equivalent under ^p (they cannot be completed to 
graphs in F by gluing any graph onto them). Therefore, 
without loss of generality, we assume that Gi, G2, . . . are 
pairwise color-compatible well-colored i-boundaried graphs. 
Now, for each pair Gi, Gj, there is a graph H^j G W t s ™ an 
such that Gi® c Hij G F but Gj® c H itj £ F. From 
Gi © c Hi j G F, it follows that Hij is a well-colored t- 
boundaried graph that is color-compatible with G;. Now, 
we have H(Gi) ® h H(H itj ) = H(Gi® c H id ) G H(F). 
Moreover, H(Gj) ® h H(H id ) = U{G j ® c H i , j ) $ H(F). 
That is, H(Gi) ^uiF) H{Gj) and therefore has 
infinite index. 

Assume that ^-h{f) fi &s infinite index over 'H(W|™ an ). 
Then, there is a family H{G 1 ),H(G 2 ),H(G 3 ), . . . of hy- 
pergraphs in T-l{Uf^ aVi ) that are pairwise inequivalent un- 
der r*>p. Since there are only a finite number of parti- 
tions of t labels into hyperedge-labels and vertex-labels, 
there is an infinite number of pairwise gluable hyper- 
graphs among H(Gi),H(G 2 ), Therefore, without 

loss of generality, assume that all these hypergraphs are 
pairwise gluable. Now, for each pair H(Gi), "H(Gj), 
there is a hypergraph Ui^H,^) G U(U^ &n ) such that 



"H{Gi)@iH{Hij) G H(F) but H(G 3 )® h H(H ZJ ) <£ H(F). 
Hence, since "H(Gj) ©h = ©c Hij), we have 

Gi © c Hi j G F. However, with the same reasoning, 
Gj ffi c Hij ^ F. Hence, ~p has infinite index over F. □ 

We can use Theorem 3 to constructively derive algo- 
rithms for deciding properties of hypergraphs of inci- 
dence treewidth at most t: assume that we have a family 
of t-boundaried hypergraphs J- C %J argc with incidence 
treewidth at most t and such that has finite index 
over H\ aisc . There is a set F C W' a 2 lgc for which T = H(F). 

We can decide in linear time whether a given hyper- 
graph H G H\ arsc is isomorphic to a hypergraph in J-: 
compute the incidence graph G of H, that is, H(G) = H . 
Since the graph G has treewidth at most i, we can compute 
a tree decomposition for G in linear time [3]. In the same 
way as in [7, Theorem 6.72], this tree decomposition can 
be converted in linear time into a parse tree T over the 
operators in Definition 2 that generates a graph isomorphic 
to G. By Theorem 3, a finite tree automaton can check 
whether T generates a graph G' isomorphic to a graph 
in F, which is the case if and only if H(G') is isomorphic 
to some graph in T ' . The finite tree automaton can be 
constructed from the equivalence classes of ^jf. 

5 Hypergraph Cutwidth is FPT 

In this section we show that Hypergraph Cutwidth 
is fixed-parameter linear. By Lemma 1, this implies that 
Trellis Width is fixed-parameter linear as well. 

Let fc-HCW denote the class of all hypergraphs with 
cutwidth at most k. We will use Theorem 3 to show that 
the parse trees for graphs in fc-HCW can be recognized by 
a finite tree automaton. To make Theorem 3 applicable, 
we first show that, for the hypergraphs in /c-HCW, we can 
find a constant upper bound t on their incidence treewidth. 
This implies that isomorphic graphs can be generated by 
linear-size parse trees over the operator set in Definition 2 
or, in terms of Theorem 3, that the graphs in fc-HCW are 
isomorphic to the graphs in a subset of C ^(W® 2 ). 

Lemma 3. Let H = (V, E) be a hypergraph. If H has 
cutwidth at most k, then H has incidence treewidth at 
most max{fc, 1}. 

Proof. Suppose H has cutwidth at most k. Let H' = 
(V, E') denote the hypergraph obtained from H by remov- 
ing all hyperedges of size at most 1. Consider a linear lay- 
out / of cutwidth at most k of the vertices of H' . Without 
loss of generality, assume that I maps to the natural num- 
bers 1 to n and let V = {v\, . . . , v n } be such that l{vi) = i. 
We construct a path decomposition for the incidence 
graph l(H') with the bags L\, Ri, L%, R%, L„_i,i?„_i 
and a path connecting the bags in this order. For ev- 
ery i, 1 < i < n — 1, let Li :— 9i t H>(i) U {vi} and 
Ri '■= Qi,H'{i) U {v i+ i}, recalling that Oi,H'{i) is the set 
of hyperedges that cross position i. 

We now prove that this is a path decomposition for l(H'). 
Let {vi, e} be any edge in the incidence graph of H' with 
Vi G V and e G E' . Since |e| > 2, we have that either 
e G Qi,H f {i — 1) or e G Oi,H'(i)- Therefore, either e G Ri-i 
or e G Li. But u,; G Ri-iDLi. Thus, Vi and e occur together 
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(a) A 3-boundaricd hypergraph H(G). 
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(b) A 3-boundaried hypergraph 'H(H). 
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(c) The glued hypergraph H(G) e h H(H) = H(G® C H). 



Figure 2: The two hypergraphs represented by the well-colored 3-boundaried graphs G and H in Figure 1 and the 
glued hypergraph H(G) ®^%(H), which is equivalent to %(G® C H). 



in at least one bag. Next, consider the vertex Uj £ V . It 
occurs in the two bags R4-1 and L,, which are consecutive 
and thus induce a connected path. Finally, consider a 
hyperedge e G E' . It occurs in all bags Li,Ri, . . . , Lj,Rj 
where Vi is the first vertex in the layout I occurring in e 
and Vj is the last vertex in I occurring in e. But these 
bags are all consecutive on the path and thus induce a 
connected path. 

The width of this path decomposition is, by construction, 
ma3ii<i< n -i{\6H' < k. To obtain a tree decomposi- 
tion for H from this path decomposition for H' , we only 
need to take care of hyperedges of size at most 1. For every 
hyperedge e € E of size 1, add a new bag {e, v}, where v is 
the unique vertex contained in e, and make it adjacent to 
an arbitrary bag containing v. For every hyperedge eeE 
of size 0, add a new bag {e}, and make it adjacent to an 
arbitrary bag. In this way, we obtain a tree decomposition 
for the incidence graph of H of width at most max{fc, 1}. 
Thus, H has incidence treewidth at most max{fc, 1}. □ 

Having Lemma 3 and Theorem 3 at our hands, it now re- 
mains to prove that the canonical right congruence ^fc_ncw 
of fc-HCW has finite index. This finally shows that fc- 
HCW can be recognized in linear time and, therefore, that 
Hypergraph Cutwidth is fixed-parameter linear. 

To show that ~&-hcw has finite index, we show that, 
given a t-boundaried hypergraph G, only a finite number 
of bits of information about a t-boundaried hypergraph H 
is needed in order to decide whether G®^H G fc-HCW. 
To this end, we employ the Method of Test Sets [7] : let T 
be a set of objects 3 called tests. A t-boundaried graph can 
pass a test. For t-boundaried hypergraphs G\ and Gi, let 
Gi ~y G2 if and only if G\ and G2 pass the same subset of 
tests in T . Obviously, ~y is an equivalence relation. Our 
aim is to find a set T of tests such that ~y refines ~fc_HCW 
(that is, if Gi ~y G2, then Gi ~jt_HCW G2). This will 
imply that, if ~y has finite index, so has ~tHcw- To 
show that ~y has finite index, we show that we can find a 
finite set T such that ^y refines ^k-ncw- 

Intuitively, in our case, a hypergraph G will pass a 
test T if it has a restricted linear layout, where each of its 
boundary vertices gets mapped to predefined integer values 
and each of the remaining vertices "lands" within one of a 
set of given "landing zones" between the integer values of 

3 For the moment, it is not important what exactly a test is. 



the real line. This restricted linear layout will impose the 
same restrictions on an optimal cutwidth layout for G that 
are also imposed by an optimal cutwidth layout of G ©h H 
for some hypergraph H that corresponds to T . 

Definition 7. Let G and H be t-boundaried hypergraphs 
such that G(BhH E fc-HCW. Let denote the vertex 
that is mapped to position i in an optimal cutwidth layout 
that maps to integer values. 

A landing zone is a tuple in {0, ... ,k} x 2* 1 '—*}. A 
size-n test T = (tt,S) consists of a map tt : {1, . . . , t} — > 
{1, . . . , n} and a sequence S = (So, Si, ... , S n ) of landing 
zones. 

We define an H-test T = (tt, S) as follows: for each 
vertex Vi € d(H), set w(£) := i, where I is the label of 1^. 
For i € {0, . . . , n}, Si := (wi, Ei), where 

i) u>i is the number of hyperedges in H and that contain 
vertices in {v%, . . . , Vi} n V(H) and . . . , v n } n 
V(H). 

ii) Ei is the set of labels of hyperedges in H containing 
vertices in {v\, . . . , v{\ n V(H) and . . . , v n } H 
V(H). 

Figure 3 illustrates this definition. We now formally define 
what it means to pass a test. The definition models the 
aim that if a graph G passes an iJ-test, then G(BhH 6 
fc-HCW. 

Definition 8. Let G = (V, E) be a t-boundaried hyper- 
graph and T = (tt, S) be an iJ-test for some t-boundaried 
hypergraph H, where S = (So, ■ ■ ■ , S n ) and Si — (u>i,Ei). 

A T '-compatible layout for G is an injective func- 
tion f : V M. such that each vertex v g d(G) with label I 
is mapped to tt(£) and such that every vertex v EV\ d(G) 
is mapped into some open interval (i, i + 1) for < i < n. 

The weighted cutwidth of f is maxi£%_(\9f(i)\ + wu\), 
where 0f(i) is the set of hyperedges containing two ver- 
tices v and w with f(v) < i < f(w) and that do not have 
a label in Ey^ . 

Finally, G passes the test T if there is a T-compatible 
layout / for G whose weighted cutwidth is at most fc. 

Lemma 4. For T being the set of all tests, the equivalence 
relation ~y refines ~fc-HCW- 

Proof. We show that if two t-boundaried hypergraphs 
Gi,Gz pass the same subset of tests of T, then, for all 
t-boundaried hypergraphs H, G\ ©h H £ fc-HCW if and 
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Figure 3: Construction of the ff-test illustrated using the glued hypergraph ff(G) ffih H(H ) from Figure 2. 



only if G 2 ffih ff e fc-HCW. The proof is based on the 
following two claims that are proved afterwards. 

i) If Gi ffi h ff € fc-HCW, then Gi passes the ff-test. 

ii) If G 2 passes the ff-test, then G 2 ffihff € fc-HCW. 

Let ff be a i-boundaried hypergraph such that 
Gi© h ff G fc-HCW, and let T be an ff-test. By (i), 
Gi passes T. Since Gi and G 2 pass the same tests, also 
G 2 passes T. By (ii), it follows that G 2 ffi h ff G fc-HCW. 
The reverse direction is proved symmetrically. It only 
remains to prove (i) and (ii). 

(i) Let Z be a layout of Gi ffih ff with cutwidth at most fc 
that only maps boundary vertices to integral positions and 
let T be the ff-test. The restriction / of I to the vertex 
set of Gi is then clearly T-compatible. Moreover, / has 
weighted cutwidth at most fc: to this end, for an arbi- 
trary i € N, we show that + w uj from Definition 8 
is at most fc. Consider the set A of hyperedges of Gi ffih ff 
that contain two vertices v,w with f(v) < i < f(w). 
Obviously, \A\ < fc. We show \0 f (i)\ + = \A\. The 
set A can be partitioned as A = A H l±l A Gl l±l A*, where 
A are hyperedges in ff, A Gl are hyperedges in Gi 
and A* are hyperedges that are neither in ff nor in G±. 
The hyperedges in A* are labeled edges that are the 
result of taking the unions of equally-labeled hyperedges 
from Gi and ff. We further partition A* = A + W A~ 
such that A + is the set of hyperedges with labels in £Jui . 
Now, by definition, we have luuj = + |j4 + | and 
1^(41 = \A Gl \ + \A~\. Therefore, \6 f (i)\ + io Li j = |A| < fc. 

Note here, that, if we were to define 9f(i) in Definition 8 
to contain all edges that contain vertices left and right 
of i (rather than only those with no label in Eu \ ) , then 
we would obtain \6 f (i)\ = \A Gl \ + \A + \ + \A~\ here, that 
is, + W[_ij would count the edges in A + twice. 

(ii) The ff-test T was obtained from a linear layout I of 
cutwidth fc for Gi ffih H. Moreover, there is a T-compatible 
layout / for G 2 . First note that / and / agree on the 
layout of vertices in <9(G 2 ) and d(H) and that, apart from 
these, / lays out vertices at non-integral positions, whereas 
I lays out vertices at integral positions. Because of this, 
in a layout g for G 2 ©h ff that lays out vertices v of ff at 
position l(v) and vertices v of G 2 at position f(v), every 
two vertices in G 2 ©h ff are laid out at distinct positions 
by g. Hence, g is injective and, therefore, a layout. 



Now it is easy to observe that g is a layout of cutwidth 
at most fc for G 2 ffih H: to this end, for a position i € E, 
consider the set A of hyperedges of G 2 ffih ff containing 
vertices v, w with g(v) < i < g(w) and let it be partitioned 
A = A H l+l A° 2 W A + l+l A" in the same way as above. Again, 
we have io w = \A H \ + \A+\ and \0 f (i)\ = \A G *\ + \A~\, 
yielding \A\ = \A G *\ + \A H \ + \A+\ + \A~\ = 9 f (i) + W[ii . 
Since G 2 passes the ff-test, this is at most fc. □ 

We have shown a set of tests T such that re- 
fines ~fc_HCW- However, the set T is infinite and, therefore, 
does not yet yield that ~tncw has finite index. However, 
we can achieve this using the following lemma. 

Lemma 5. Let G and ff be i-boundaried hypergraphs. 
For every ff-test T 1; there is a test T 2 of size 2t(2k + 2) 
such that G passes T\ if and only if it passes T 2 . 

Proof. Let the sequence S of landing zones of T\ be So = 
(E ,w ) : ...,(E n ,w n ) = S n . For E C {1, . . . , t}, we call 



a maximal subsequence (Ei,Wi), . . . , (£j 



Wi 



of 5 with 



^ = 



= Ei a strait. We first show that there are at 



most 2t straits, and then show that we can shorten each 
strait to a length of 2fc + 2 by removing some landing zones 
without changing the satisfiability of the test. 

For £ C {1, . . . , t}, let h •= {i < n : I G Ei}. Observe 
that I( is a consecutive subset of {0, ... , n}: this is by the 
construction of the Ei in Definition 8 from an optimal lay- 
out for some i-boundaried graph G' ffih ff • That is, each Ig 
for some £ € {1, . . . , t} is an interval of the natural numbers 
with a minimum element and a maximum element, which 
we both call events. Hence, the Ig for all £ £ {1, . . . ,t} in 
total have at most 2t events. Since straits can only start at 
an event, and since only one strait can start at a fixed event, 
it follows that S is partitioned into at most 2t straits. 

It remains to shorten the straits. Let (Ei,Wi), . . . , 
(Ej,Wj) be a strait. We apply the data reduction rules 
(R1-R3) from the proof of Theorem 6.82 by Downey 
and Fellows [7], which are based on the following obser- 
vations: deleting from S one of two consecutive landing 
zones {E u Wi), (E i+1 ,w i+1 ) with E l = E i+1 and Wi = w i+1 
yields an equivalent test, as does adding such landing 
zones. Moreover, if we replace a landing zone (Ei,Wi) by 
a landing zone (Ei^w'A with w\ < Wi, we obtain a test 
that is easier to pass. 
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Since the Wi are, by construction of TY, bounded from 
above by fc, we can transform T\ into Ti by these data 
reduction rules so that each of the 2i straits has length 
at most (2k + 2) [7, Theorem 6.83]. □ 

Now the following theorem is easy to prove. 

Theorem 4. Hypergraph Cutwidth is fixed-parameter 
linear. 

Proof. Lemma 3 shows that graphs in fc-HCW have con- 
stant treewidth at most t, and therefore, that all such 
hypergraphs can be linear-time transformed into linear- 
size parse trees for i-boundaried hypergraphs. Using The- 
orem 3, we show that parse trees corresponding to hy- 
pergraphs in k-HCW are recognizable by a finite tree 
automaton: let T be the set of all tests and T' be the 
set of all tests of size 2i(2fc + 2). Lemma 5 shows that 
~7-/ refines Lemma 4 shows that ^j- refines ~fe-HCW- 
Therefore, ~fc-HCW has at most as many equivalence classes 
as r^ij-i. Since k and t are constant, T' is finite, implying 
finite index for and, consequently, for ~^hcw- d 

6 Hypertree Width and Variants 

In this section we show a negative application of our hy- 
pergraph Myhill-Nerode analog to Hypertree Width, 
Generalized Hypertree Width, and Fractional 
Hypertree Width. The problems are, given a hyper- 
graph H and an integer k as input, to decide whether H has 
(generalized, fractional) hypertree width at most k. Since 
Hypertree Width is W[2]-hard [19], Generalized Hy- 
pertree Width is paraNP-hard [20] (NP-hard for k = 3), 
and the parameterized complexity of Fractional Hyper- 
tree Width is open but expected to be paraNP-hard [26] , 
it would be nice to find non-standard parameters for which 
these problems are fixed-parameter tractable. While it is 
known that the generalized hypertree width of a hyper- 
graph is at most the incidence treewidth plus one [11], 
the incidence treewidth may be arbitrarily large even for 
hypergraphs with (generalized, fractional) hypertree width 
one, since adding a universal hyperedge to any hypergraph 
reduces its (generalized, fractional) hypertree width to 
one. This makes incidence treewidth a natural parameter 
to explore. However, the following theorem shows that 
Hypertree Width, Generalized Hypertree Width, 
and Fractional Hypertree Width do not have finite 
index. Moreover, the construction we use in the proof leads 
us to believe that the problems are W[l]-hard with respect 
to the parameter incidence treewidth. 

Theorem 5. Let k > 0. Let HTW fe (respectively, GHTW fc 
and FHTWfeJ be the family of all incidence graphs G such 
that H(G) has hypertree width (respectively, generalized 
hypertree width and fractional hypertree width) at most k. 
The canonical right congruences ~H(HTW fc ); ~«(GHTW fc ); 
and ~%(FHTW fc ) do not have finite index over T-L(U^™ al1 ) . 

Proof Outline. In this proof outline, we give a construction 
of a t-boundaried hypergraph H n with bounded incidence 
treewidth, for every n > 1. Then we outline how one 
can show that H n ©h H m has (generalized, fractional) hy- 
pertree width 4 if and only if n — m. This implies an 



infinite number of equivalence classes for the canonical 
right congruences ~h(HTW 4 ) and ~«(Ghtw 4 )- We refer to 
Appendix A for a detailed proof. 

For every n > 1, we construct a f-boundaried hyper- 
graph H n with t = 28, generalized hypertree width 4, 
and incidence treewidth at most 12. The vertex set 
of H n is V = AUBUCUDUSUTUX, where 
A = {a,y},B = {b,z},C = {c,y},D = {d,z},S = 
{si, . . . ,s 8 },T = {ti, ...,t s } and X = {xi, . . -,x 6n }. The 
hyperedge set of H n is E — {A, B, C, D} U B s U {S c , S d , 
S y ,S z } UB T U {T a ,T b , T y ,T z } U {E 3h E 3i+1 : 1 < * < 
2n} U {Ei, i+ i : I <i < 6n}, where 

Bs is the set of all possible binary hyperedges on S, 
S c = {c, 81,82}, Sd = {d, s 3 , s 4 }, 
S y = {y, s 5 ,s 6 }, S z = {z, s 7 , s 8 } 

Bt is the set of all possible binary hyperedges on T, 





= {a,ti,t 2 },T b = {b 7 t 3 ,t 4 }, 


Ty 


= {y,h,h},T z = {z,t 7 ,t 8 } 


Ei 




E 3l 


= {a, c, y, x 3i } for 1 < i < 2n, 


E31+1 


= {b.d, z,x 3i+ i} for 1 < i < 2n, 


Eqtl 




'6i+l,6i+2 


= {a, b,x m+ i ,x 6i+2 } for < i < n, 


'6i+4,6i+5 


= {c, d, x 6i+4: , xei+5} for < i < n, and 


-£'3i,3i+l 


= {x 3 i,x 3l+ i} for 1 < % < In. 



The set of boundary hyperedges is {^4, B, C, D, S C7 Sd, 
S y , S z ,T a , Tf,, T y , T z }. The set of boundary vertices is SUT. 
They are labeled from 1 to 28 in this order and by increas- 
ing indices. See Figure 4 for an illustration of H2 induced 
on V \ (SUT). 

Consider a tree decomposition for H n (Bh H m with (gen- 
eralized) hypertree width 4. The vertex sets S and T 
and the hyperedges containing them make sure that some 
bag Bs of the decomposition contains all of S and that 
some bag Bt contains all of T. Now, both in Q(H n ) and 
in Q(H m ), there is a path from a vertex in S to a vertex 
in T passing through all vertices Xi by increasing indices. 
The edges of this path are covered by intermediate bags 
lying on the path from Bs to Bt on the tree decomposi- 
tion. Observe that no vertex Xi is contained in a boundary 
hyperedge. Therefore, when we restrict the tree decomposi- 
tion to the vertices in H n , we recover a tree decomposition 
for H n where all intermediate bags are covered by at most 
three hyperedges. Moreover, our construction makes sure 
that when a bag is covered by three hyperedges, at least 
two of them are boundary hyperedges. In every such tree 
decomposition for H n , when considering the intermediate 
bags starting from Bs that contain either A, B or C, D 
in their cover, we first encounter bags covered by C, D, 
then bags covered by A, B, then bags covered by C, D, 
and so on, and there are exactly n alternations from C, D 
to A, B in this sequence. Therefore, in order to be able 
to merge such decompositions for H n and H m , we must 
have n = m. □ 

The number of equivalence classes observed in Theorem 5 
entails a lower bound on the amount of information that 
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Figure 4: The incidence graph of H2 induced on V \ (SUT). 
Boxes represent hyperedges. 

needs to be maintained by an algorithm when it decides 
whether a given hypergraph has (generalized, fractional) 
hypertree width k using a tree decomposition of the inci- 
dence graph: typical such algorithms implicitly remember 
for each bag of the tree decomposition (by means of a 
dynamic programming table or by the states of a tree 
automaton) which equivalence classes of the bag under 
consideration can complete into a graph with (generalized, 
fractional) hypertree width k. 

However, restricting the construction in the proof of 
Theorem 5 to at most n vertices, the proof of Theorem 5 
exhibits a class C of i-boundaried hypergraphs on at most 
n vertices with constant incidence treewidth and constant t 
for which the canonical right congruences have Q(n) equiv- 
alence classes. Now, consider a class C of 0(fc)-boundaried 
hypergraphs where each hypergraph contains k copies of 
hypergraphs from C and has at most n' vertices. Then, 
the number of equivalence classes of the canonical right 
congruences is £l((n' / k) k ) for C. Hence, we conjecture that 
an algorithm with running time f(k) ■ n c for a constant c 
and / only depending on k does not exist. 

Conjecture 1. Hypertree Width, Generalized Hy- 
pertree Width, and Fractional Hypertree Width 
are W[l]-hard with respect to the parameter incidence 
treewidth. 

7 Conclusion 

We have extended the graph analog of the Myhill-Nerode 
theorem to colored graphs and hypergraphs, making the 
methodology more widely applicable. Our positive ap- 
plication shows that Hypergraph Cutwidth, Trellis 
Width, and Matroid Pathwidth are fixed-parameter 



linear. As a negative application, we showed that Hyper- 
tree Width, Generalized Hypertree Width, and 
Fractional Hypertree Width do not have finite index, 
and therefore the parse trees associated to Yes-instances 
of bounded incidence treewidth cannot be recognized by 
finite tree automata. 
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A Proof of Theorem 5 

We continue the proof of Theorem 5. Recall the construc- 
tion of H n from the proof outline. 

In this construction, the vertices in S and T and the 
hyperedges containing them are only used to make sure 
that every tree decomposition of H n with (generalized, 
fractional) hypertree width 4 contains a bag B~\ with 
the vertices S U {c, d, y} and a bag Bq u +i with the 
vertices T U {b, y, z}. Since the sets S U {c,d,y,z} 
and T U {a, b, y, z} can also be covered by four hy- 
peredges, all of which are boundary hyperedges, let 
V = ({Xj :i£l},T) be a tree decomposition for H n with 
the bags B-\ = SU{c, d, y, z} and i?6n+i = {a, b, y, z}. 
We observe that all other vertices of H n occur in bags 
that are in the same connected component of the forest 
obtained from T> by removing these two bags. 

Claim 1. The tree decomposition T> contains a bag Bi, < 
% < 6n, with {s s ,xi,c,d,y,z} C B , {ti, x 6n , a, b, y, z} C 
Be n , and {a, b, c, d, y, z, Xi, Xi+{\ C Bi, for every i, 1 < i < 
6n. 

Proof. The primal graph G(H n ) contains the 
cliques {s s ,xi}, {a,b,x 1 ,x 2 \, {a, b, x 2 , x 3 }, {a,c,y,x 3 }, 
{x 3 ,x 4 }, {xi,b,d,z}, {c,d,X4,x 5 }, {c,d,x 5 ,x 6 }, 
{a,c,y,x 6 }, {x 6 ,x 7 }, {b,d,z,x 7 }, {a, b, x 7 , x s }, 
{xQ n ,ti}, and every two consecutive cliques in this 
list intersect in at least one vertex. In particular, 
we observe the path (sg, x\, x 2 , ■ ■ ■ , Xe n , ti) in Q(H n ). 
Thus, V contains bags Bq 2 {ss,xi}, Ben 2 {ti,X6n}, 
and Bi 3 {x.i, Xi+i}, 1 < i < 6n. Moreover, each Bi, 
< i < 6n, contains c,d,y,z since B-\ contains c,d,y,z, 
Ben-i contains c, d, £>6n+i contains y, z, and without loss 
of generality, we can assume the Bi, < i < 6n, were 
chosen such that they are on the path from B-\ to £>6n+i 
in T. Similarly, each Bi, 1 < i < 6n, contains a, b. □ 

A tree decomposition for H n is a good tree decomposition 
if it contains the bags B-\ = S U {c, d, y, z} and Be n +i = 
TU{a, b, y, z} and every bag except B_\ and B§ n +\ can be 
covered with at most three hyperedges, and in case such a 
bag is covered with exactly three hyperedges, two of these 
hyperedges are in the boundary. A good cover for a good 
tree decomposition is a cover for each bag according to the 
specifications of a good tree decomposition. 

Claim 2. If V is a good tree decomposition for H n , then, 
for every i, — 1 < i < 6n, there is a path from the bag Bi to 
the bag Bi + i that avoids all the bags Bj, j G {—1, . . . , 6n + 

1}\{M + 1}. 

Proof. Suppose the path from Bi to Bi+\ passes through Bj 
with j 6 {—1, • • • , 6rt + 1} \ {i, i + 1}. Since every bag on 
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denote the restriction of T> to H, i.e., it has the same tree, 
but each bag is restricted to the vertices of H . 

Claim 7. T>\ H is a good tree decomposition for H. 

Proof. Consider a bag B of T> besides B-\ and Bq u +i- The 
bag B contains a copy of some Xi from H' . This vertex is 
covered by some hyperedge from H' that does not belong 
to the boundary. Therefore, B\ H is covered by at most 
three hyperedges. Suppose B\ H is covered by exactly three 
hyperedges. Then, the cover of B\ , contains at most one 
hyperedge that does not belong to the boundary. But, since 
each such hyperedge covers at most two vertices among 
{a, 6, c, d}, the cover of B contains at least two boundary 
hyperedges. This proves the claim. □ 

Symmetrically, "D\„, is a good tree decomposition for H' . 
Since D\ H and Z>i , have the same signature, we conclude 
that n = m due to Claim 6. This proves that the canonical 
right congruence ^-h(ghtw 4 ) does not have finite index 
over H(U^ an ). 

To see that the same proof applies to ~h(htw 4 )i observe 
that in our construction every hyperedge covering a bag is 
a subset of that bag. To see that it extends to ~«(fhtw 4 ); 
observe that for every bag Bi, < i < 6n, an optimal 
fractional cover is integral, and Claim 4 can be extended 
to A, B G Q with weight 1 — similarly for Claim 5. This 
completes the proof of Theorem 5. 

Proof. Consider a bag B on P^i+3 and let Q 2 {A, B} be 
a cover for B. The bag B contains the intersection of two 
bags that are consecutive in the list Bi,Bi+\,Bi+i,Bi+3. 
Therefore, at least one of x i+ i, x i+2 , x i+3 is in B. We also 
have that c,d G B since c, d G Bi fl B i+3 . However, no hy- 
peredge contains a^+i, c, d or Xi+2, c, d or Xi+3, c, d. Thus, 
\Q\ > 4, and therefore Q is not part of a good cover. □ 

Claim 5. For every i G {3,9, 15, ••• , 6n — 3}, no bag 
on Pi.i+3 is covered by a set of hyperedges Q with C,D G Q 
in a good cover. 

Proof. The proof is symmetric to the proof of Claim 4. □ 

Consider a good cover of T>. A switch is an inclusion-wise 
minimal subpath (Yi, . . . , Yj) of the backbone of V where 
Yi is covered by Qi with C,D G Qi and Yj is covered by Qj 
with A,B£ Qj . The signature of a good cover of V is its 
number of switches. 

Claim 6. Each good cover of each good tree decomposition 
of H n has signature n. 

Proof. The claim follows from Claims 3, 4, and 5. □ 

Due to Claim 6, we can speak of the signature of H n and 
the signature of a good tree decomposition of H n as the 
signature of some good cover of such a tree decomposition. 

Let H = H n and H' = H m . Consider a tree decompo- 
sition V = ({Xi : i e I},T) of H® h H' with generalized 
hypertree width 4. Without loss of generality, suppose 
the bags B-\ = S U {c, d, y, z} and Be n +i — T U {a, b, y, z} 
are leafs of this decomposition and their neighboring bags 
contain both copies of x\ and xe n , respectively. Let T>\ H 



the path from Bi to Oj+i contains Bi fl Oi+i, we have 
that Xi+\ G Sj. But then {s%, x±, c, d, y, z, Xj+i} C £>j 
(if j = 0) or {ti,x 6n ,a, b,y, z.x. l+1 } C Bj (if j = 6n) or 
{a, 6, c, d, y, z, x } ■, Xj + i, Xi + i} C (otherwise), implying 
that -Bj cannot be covered by two hyperedges and it cannot 
be covered by three hyperedges of which two are in the 
boundary. □ 

Claim 3. In every good cover, Bq is covered by {E\, C, D}, 
Bq 71 is covered by {Eq u , A, B}, and for every «,!<«< 6n, 



Bi is covered by 



{Ei t i+i,C, D} 


if i 


= 1 


(mod 6), 


\Ei,i+i,C, D} 


if i 


= 2 


(mod 6), 


( {Ei,E i+1 } 


if i 


= 3 


(mod 6), 


B} 


if i 


= 4 


(mod 6), 


B} 


if i 


= 5 


(mod 6), 


.{-Sj^i+i} 


if i 


= 


(mod 6). 



Proof. The claim easily follows from Claim 1. 



□ 



Suppose T> is a good tree decomposition for H n . The back- 
bone of T> is the path P in T starting at the bag 
and ending at the bag Bg n+ i. By Claim 2, P visits 



Bo, Si 



1 B^n in this order. Let Pi j denote the subpath 



of P starting at Bi and ending at Bj. 

Claim 4. For every i G {0,6, 12, ••• , 6n — 6}, no bag 
on Pi y i+3 is covered by a set of hyperedges Q with A,B G Q 
in a good cover. 
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